Appln.No.: 10/021,255 
Amendment dated February 22, 2005 
Reply to Office Action of January 14, 2005 

REMARKS/ARGUMENTS 

The office action of January 14, 2005 has been carefully reviewed and these remarks are 
responsive thereto. Reconsideration and allowance of the instant application are respectfully 
requested. Claims 1-41 remain pending in this application. Claim 42-46 have been canceled 
without prejudice or disclaimer. 
SECTION 102 REJECTIONS 

Claims 1-9, 12-26 and 29-34 stand rejected under 35 U.S.C. § 102(b) as being anticipated 
by U.S. patent no. 5,865,464 to Bhandari et al. ("Bhandari"). Applicants respectfully traverse this 
rejection. 

Claims 1-8 and 18-25 

The action continues to allege that Bhandari discloses all the features of independent 
claims 1 and 18. In the last response, applicants submitted that Bhandari failed to teach or 
suggest the claim 1 and 18 features of identifying media objects stored in the database that are 
related to the captured media object and inferring organization information for the media object 
based upon information obtained from each of the stored media objects that are related to the 
[captured] media object as well as organizing the media object in the database based upon the 
inference. 

Refuting applicants' position, the action with reference to col. 10, lines 3-17 of Bhandari 
contends that "comparing the query to an archival multimedia object description" shows the 
claimed feature identifying media objects stored in the database that are related to the captured 
media object. Also, the action with reference to col. 6, lines 18-24 and col. 10, lines 3-17 of 
Bhandari alleges that "comparing the query to an archival multimedia object description in order 
to find a match" shows inferring organization information for the media object based upon 
information obtained from each of the stored media objects that are related to the [captured] 
media object as well as organizing the media object in the database based upon the inference. 
Applicants respectfully disagree. 

The action appears to merge the disclosures in Bhandari of the two independent 
operations of inputting data to be associated with an image using natural language and querying 
the database for searching and retrieving desired images using natural language. With reference 
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to Figs. 2-4, Bhandari describes a software program "for inputting data that is to be associated 
with an image using natural language." Col. 3, lines 5-7. According to Bhandari , an image is 
captured; a user inputs description fields and a caption to be associated with the image; the 
caption undergoes natural language processing to produce the frame representation of the 
caption; the caption frame is cataloged in a frame database; and the image is stored in the image 
database. Col. 4, lines 26-43. The specifics of natural language processing to produce a frame are 
described col. 4, line 47 to col. 6, line 39. Nowhere does Bhandari teach or suggest that this 
process involves the claim 18 features of identifying media objects stored in the database that are 
related to the captured media object and inferring organization information for the media object 
based upon information obtained from each of the stored media objects that are related to the 
media object and organizing the media object in the database based upon the inference. 

With reference to Fig. 5, Bhandari describes the software program for searching and 
retrieving desired images using natural language processing. As is clear, this process is 
performed after the frame database is populated with frames and is not part of inputting images. 
In this process, a user inputs a natural language query to search for an image. The natural 
language input is processed and then the frame database is searched using the frame 
representation of the query to determine if any frames in the frame database match. Col. 6, lines 
41-49. Frames which match are then presented to the user on a display for permitting the user to 
view the images. Col. 7, lines 49-52. According to Bhandari, if the user finds the images 
acceptable, he may print, order, transmit, store in an album or further manipulate the retrieved 
images. Plainly, this process is wholly independent of inputting images into the database. 

Referring to the specific areas relied on by the action, col. 10, lines 3-17 are part of a 
claim directed to "retrieving a multimedia object" in the context of searching and retrieving 
desired images as shown in Fig. 5 and found in col. 6, lines 41-49. The reference to comparing 
the query roles to the archived multimedia object description roles to identify matching roles is 
done solely to retrieve the image and nothing more. Tellingly, the query is input by a user and is 
not associated with an image which is being input. 

Distinctly, claims 1 and 18 organize the media object based on the inference. That is, 
organize the media object based on the organization information for the media object based upon 
information obtained from each of the stored media objects that are related to the media object. 
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In view of the foregoing Bhandari is defective in several respects. Accordingly, 
independent claims 1 and 18 are patentably distinct from Bhandari for at least the above reasons. 
Claims 2-8, which ultimately depend from claim 1, and claims 19-25, which ultimately depend 
from claim 18, are patentably distinct from Bhandari for the same reasons as their ultimate base 
claim and further in view of the novel and non-obvious features recited therein. 
Claims 9 and 26 

The action continues to allege that Bhandari discloses all the features of independent 
claims 9 and 26. In the last response, applicants asserted, inter alia, that Bhandari lack a teaching 
or suggestion of comparing the date with threshold date information and identifying media 
objects stored in the database that are related to the media object based upon the comparison as 
recited in claims 9 and 26. Countering applicants' arguments, the final office action contends 
that Bhandari "teaches description fields and captions that are associated with each metadata, 
which include when the image was taken, creating a matching weight which is compared to a 
predetermined threshold weight" pointing to col. 4, lines 30-35, col. 6, lines 54-67, and col. 7, 
lines 1-5. Final Office Action, pages 25-26. Applicants respectfully disagree as set forth below. 

According to Bhandari at col. 4, lines 30-40, a user inputs "description fields and caption 
to be associated with a particular image (metadata)." Col. 4, lines 30-31. Admittedly, the 
description fields can include when the image was taken. A caption can include "one or more 
natural language phrases or sentences without any grammar restrictions describing any 
characteristics of the image." Col. 4, lines 35-37. Importantly, nowhere does Bhandari teach or 
suggest that the caption includes when the image was taken. After the caption is input, it is 
processed in step S6 using natural language processing with the output result being the frame 
representation for the caption. Col. 4, lines 38-40. Notably, the caption is processed and not the 
description fields including when the image was taken. 

In performing a search according to Bhandari , "the frame representation of the query is 
first matched role to role with S34 with the caption frames in the frame database." Col. 6, lines 
54-56. Thereafter, the matching results are weighted and the number and score of matched 
frames are analyzed against a predefined threshold. Importantly, since when the image was taken 
has no part in the caption frame, contrary to the action's assertion the results and their weighting 
also do not incorporate when the image was taken. Thus, a comparison of Bhandari does not 
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involve comparing a date with threshold data information and identifying media objects stored in 
the database that are related to the media object based upon the comparison as recited in claim s9 
and 26. 

Moreover, claims 9 and 26 each call for inferring organization information for the media 
object based upon information obtained from each of the stored media objects related to the 
media object, and organizing the media object in the database based upon the inference. As 
discussed with respect to claims 1 and 18, Bhandari also lacks a teaching or suggestions of these 
features. For at least the aforementioned reasons, claims 9 are 26 patentably distinguishable from 
Bhandari . 

Claims 12-14 and 29-31 

In the last response, applicants submitted that Bhandari failed to teach or suggest 
inferring organization information for the media object based upon information obtained from 
each of the stored media objects that are related to the media object; and organizing the media 
object in the database based upon the inference as called for in claims 12 and 29. In reply, the 
action retorted that Bhandari "teaches comparing the query to an archival multimedia object 
description in order to find a match" pointing to col. 6, lines 16-24 and col. 10, lines 3-17. Final 
Office Action, p. 26 

Applicants acknowledge that Bhandari compares a user-inputted natural language query 
to frame representation of stored images to find a match. Notwithstanding, the comparing 
operation is conducted for the purpose of finding an image having characteristics desired by the 
user and not to assist in organizing media objects. Indeed, when images are found in the 
Bhandari search, they are ranked and displayed for viewing by a user. A user may select any of 
the resulting images to be printed, ordered, transmitted over network, stored in an album or 
further manipulated. Tellingly, nowhere does Bhandari teach or suggest performing any action 
on the captured media object after performing this search, such as inferring organization 
information for the media object based upon information obtained from each of the stored media 
objects that are related to the media object; and organizing the media object in the database based 
upon the inference as called for in claims 12 and 29. For at least these reasons, claims 12 and 29 
are patentably distinct from Bhandari . Claims 13 and 14, which depend from claim 12, and 
claims 30 and 31, which depend from claim 29, are considered allowable over Bhandari for the 
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same reasons as their base claim, and further in view of the advantageous features recited 
therein. 

Claims 15 and 32 

In the previous response, applicants argued that Bhandari lacked a teaching or suggestion 
of comparing a media object with media objects that are stored in the database; identifying the 
stored media objects in the database that include features in common with the media object; 
inferring organization information for the media object based upon information, obtained from 
each of the media objects including features in common with the media object, representing 
organization in the database; and organizing the media object in the database based upon the 
inference. In this regarding, applicants relied on the same arguments made with respect to claims 
12-14 and 29-31 above with respect to the features of inferring and organizing and continues to 
believe that claims 15 and 32 are allowable for at least the same reasons as those claims. 

Regarding the features of comparing a media object with media objects that are stored in 
the database; identifying the stored media objects in the database that include features in 
common with the media object, the action in reply to applicants' arguments again stated that 
Bhandari "teaches comparing the query to an archival multimedia object description in order to 
find a match" pointing to col. 6, lines 16-24 and col. 10, lines 3-17. 

Applicants acknowledge that Bhandari compares a user-inputted natural language query 
to frame representation of stored images to find a match. Yet Bhandari does not compare one 
media object with media objects that are stored in the database and merely compares a user input 
to search for an object. Necessarily, Bhandari does identify stored media objects in the database 
that include features in common with the media object as there is only a comparison of a user 
input being performed against the frame representation of the stored images in the database. As 
such, claims 15 and 32 are patentably distinguishable from Bhandari . 
Claims 16,17,34 and 35 

To the extent independent claims 16, 17, 34 and 35 have the same or similar features as 
the independent claims discussed above, the reasons differentiating those claims from Bhandari 
apply to claims 16, 17, 34, and 35 as well. Accordingly, claims 16, 17, 34, and 35 are patentably 
distinguishable from Bhandari. 
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SECTION 103 REJECTIONS 

Claims 10, 27, and 35-41 stand rejected under 35 U.S.C. § 103(a) as being unpatentable 
over Bhandari in view of Loui et al., "Software System for Automatic Albuming of Consumer 
Pictures," published by ACM Multimedia Conference, 1999 (" Loui '99 "V Claim 11 stands 
rejected under 35 U.S.C. § 103(a) as being unpatentable over Bhandari in view of Loui et al., 
"Automatic Image Event Segmentation and Quality Screening for Albuming Applications," 
published by IEEE International Conference on Multimedia and Expo, 2000 (" Loui '00 "). Claim 
28 stands rejected under 35 U.S.C. § 103(a) as being unpatentable over Bhandari in view of Loui 
'99 and further in view of Loui '00 . Applicants respectfully traverse these rejections. 
Claims 10 and 27 

In the last response, applicants submitted that the combination of Bhandari and Loui '99 
would not have taught or suggest comparing the date on which the media object was captured 
with entries in a date book as recited in claims 10 and 27. To refute applicants' position, the final 
office action states that Loui '99 in section 2.1, p.l 60, lines 1-14 shows "clustering images 
based on events, [and] in order for that to be possible the dates and events must be compared to 
each other." Final Office Action, p. 27. Applicants agree that Loui '99 shows clustering images 
based on events. Nonetheless, that alone does not mean or imply that Loui '99 teaches or 
suggests comparing the date on which the media object was captured with entries in a date book. 
At most, Loui '99 time implies that time and date information is used to perform event 
clustering. However, the fact that time and date information is used to perform event clustering 
does not inherently or expressly suggest comparing the date on which the media object was 
captured with entries in a date book. As such, for at least this reason, Loui '99 fails to overcome 
the deficiencies of Bhandari . 

Moreover, in the last response applicants argued that the combination of Bhandari and 
Loui '99 would have been improper, which the action failed to address. Bhandari does not 
describe, teach or otherwise relate to albuming as described in Loui '9 9. Indeed, Bhandari is 
directed to a "method for using natural language for the description, search and retrieval of 
multi-media objects." Bhandari , Abstract. Since Bhandari only focuses on performing natural 
language queries on the caption description, does not compare the date on which an image was 
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taken to anything and is not directed to albuming, one skilled in the art would not have modified 
Bhandari with Loui '99 as set forth in the action. 
Claims 35-41 

The action failed to address the arguments raised with respect to claims 35-41 in the last 
response. As such, applicants will repeat them here. 

Claim 35 is directed to a method of organizing media objects in a database including, 
among other features, detecting a capture time for each of the media objects to be organized; 
sorting the media objects based upon the capture time to generate a sorted list; comparing the 
capture time of each of the media objects with a reference value; and grouping the media objects 
in the database based upon the comparison. The action alleges that Bhandari discloses detecting, 
but relies on Loui '99 to disclose the remaining steps of sorting, comparing and grouping. To 
show the step of comparing the capture time of each of the media objects with a reference value, 
the action points to p. 160, section 2.1, paragraph 1, lines 3-27 of Loui '99 . 

Contrary to the action's assertion, Loui '99 neither teaches nor suggests comparing the 
capture time of each of the media objects with a reference value. Instead, Loui '99 segments 
pictures using event clustering, which can involve analyzing pictures based on time differences 
to determine whether pictures are from the same event or different events. For at least this 
reason, the combination of Bhandari and Loui '99 , even if proper, does not result in the claim 35 
invention. 

Moreover, for the same reasons set forth above with respect to claims 10 and 27, one 
skilled in the art would not have been motivated to modify Bhandari with Loui '99 in the manner 
set forth in the action. 

Claims 36-41, which ultimately depend from claim 35, are patentably distinct from the 
combination of Bhandari and Loui '99 for the same reasons as their ultimate base claim and 
further in view of the novel and non-obvious features recited therein. For example, claim 40 calls 
for creating a new collection when the capture time of any one of the media objects from the 
sorted list is not within the predetermined time period from the updated reference value. To 
purportedly show this feature, the action points to col. 7, lines 36-41 of Bhandari . Yet Bhandari 
does not perform or suggest albuming as discussed above. Moreover, the cited passage of 
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Bhandari merely describes what occurs when the results of a search are unacceptable based on a 
predefined threshold, namely a search involving expanded keywords is performed. 
Claims 11 and 28 

Applicants submit that the combination of Bhandari with either Loui '99 or Loui '00 , or 
both Loui '99 and Loui '00 does not result in the invention of claims 1 1 and 28 and would have 
been improper for substantially the same reasons set forth with respect to claims 10 and 27. 
Applicants note that the use of the term "global similarity" in Loui '00 does not provide a 
teaching or suggestion of global date book recited in claim 1 1 . 



It is believed that no fee is required for this submission. If any fees are required or if an 
overpayment is made, the Commissioner is authorized to debit or credit our Deposit Account No. 
19-0733, accordingly. 

All rejections having been addressed, applicants respectfully submit that the instant 
application is in condition for allowance, and respectfully solicit prompt notification of the same. 
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Respectfully submitted, 
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